CUDIA: Probabilistic Cross-level Imputation using Individual Side Information

Due to privacy or legal issues, aggregate data publication is a common practice in healthcare and medical research. However, to find out valuable individual level relationships from the aggregate data, many data mining algorithms suffer from the aggregation bias and the information loss, or require rather strict assumptions, which are usually unverifiable. Furthermore, even if individual level data are available, as many healthcare studies are performed with a pre-specified goal, a limited scope of variables constraints the range of the research focus. How can one run data mining procedures on such data where different variables are available at different levels of aggregation or granularity? In this paper, we seek a better utilization of variably aggregate datasets, which are possibly from different sources. By modeling the generative process of such datasets using a Bayesian directed graphical model, we propose a novel “cross-level” imputation technique. The imputation is based on the underlying data distribution and shown to be unbiased. This imputation can be further utilized in a subsequent predictive modeling, showing improved performances than just imputing the aggregate information as it is. Experimental results using a simulated dataset and the Behavioral Risk Factor Surveillance System (BRFSS) dataset are provided to illustrate the generality and capabilities of the proposed framework.